Attributed Consistent Hashing for Heterogeneous Storage System

Cloud-scale storage system is an important building block of the cloud infrastructure. It demands the flexibility to distribute data and provide high I/O performance. Consistent hashing algorithm is widely used in large-scale parallel/distributed storage systems for the decentralized design, scalability and adaptability. It can evenly distribute data among nodes but lack efficiency in a heterogeneous environment. In this research, we propose a novel data placement algorithm, which is based on consistent hashing while making it more efficient for heterogeneous storage systems. By considering both the capacity and bandwidth attributes of nodes, our algorithm can make better use of heterogeneous devices. Our current proof-of-concept evaluations with a distributed stroage system, Sheepdog, show promising results.